Papers with computational budgets
Token and Head Adaptive Transformers for Efficient Natural Language Processing (2022.coling-1)
Copied to clipboard
| Challenge: | Pre-trained language models like BERT have shown significant accuracy improvements on various tasks, but their computational cost and memory footprint are prohibitive. |
| Approach: | They propose to extend Length Adaptive Transformer to extend the model to a token and head pruning scheme to optimize pruning efficiency. |
| Outcome: | The proposed model can compress and accelerate BERT-based models by fine-tuning and a token and head pruning scheme. |
Sticker-TTS: Learn to Utilize Historical Experience with a Sticker-driven Test-Time Scaling Framework (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large reasoning models have exhibited strong performance on complex reasoning tasks, but current test-time scaling methods rely on redundant sampling and ignore historical experience utilization. |
| Approach: | They propose a test-time scaling framework that coordinates three collaborative LRMs to iteratively explore and refine solutions guided by historical attempts. |
| Outcome: | The proposed framework surpasses strong baselines on three mathematical reasoning benchmarks, including AIME-24, AIME-25, and OlymMATH. |